Two-Stages Support Vector Regression for Fuzzy Neural Networks with Outliers

guaranteed to find a global extremism. Another, the lease square (LS) versions of SVM, called as LS-SVM, is also investigated for classification [3] and regression (LS-SVMR) [4]. In these LS-SVM formulations on works with equality instead of inequality constraints and a sum squared error cost function as it is frequently used in the training of traditional neural networks. This reformulation greatly simplifies the problem in such a way that the solution is characterized by a linear system, more precisely a KKT (Karush-Kuhn-Tucker) system [5], which takes a similar from as the linear system that one solves in every iteration step by the interior point method for the standard SVM [6]. This linear system can be efficiently solved by iterative methods such as conjugate gradient [7]. Then, a LS-SVM is computationally more effective than a SVM [8]. While a LS-SVM incorporates all training data in the network to produce the result, the traditional SVM selects some of SVs that are important in the regression. This sparseness of the traditional SVM can also be reached with the LS-SVM by applying a pruning method [8]. Besides, a radial basis function network is one of the SVM and it can be found in [2]. That is, the support vector algorithm can automatically determines cents, weights, and threshold that minimize an upper bound on the expected test error for the radial basis function network. In this study, two-stages support vector regression (TSSVR) approach is proposed to deal with training data set with outliers for fuzzy neural networks (FNNs). The proposed approach in the stage I, called as data preprocessing, the support vector machines for regression (SVMR) approach is used to filter out the outliers in the training data set and determine the number of fuzzy rule. Due to the outliers in the training data set are removed, the concept of robust statistic theory have no need to reduce the outlier’s effect. Then, the training data set except for outliers, called as the reduced training data set, is directly used to training the sparse least squares support vector machines for regression (LS-SVMR) in the stage II. Consequently, the learning mechanism of the proposed approach for fuzzy neural network does not need iterated learning for simplified fuzzy inference systems. Based on the simulation results, the performance of the proposed approach is superior to the robust LS-SVMR approach when the outliers are existed.